Message Type Extraction Based Alert Detection in System Logs

The task of alert detection in event logs, i.e. determining which events in the event log require action from an administrator, is very important in preventing or recovering from downtime events. The ability to do this automatically and accurately provides significant savings in time and cost of downtime events. In this work we combine message type extraction based alert detection with the entropy based approach of the Nodeinfo algorithm, which is in production use at Sandia National Laboratories, to significantly improve its performance. We show that with Message Type Indexing (MTI) and some modifications to the Nodeinfo framework, we can achieve an ∼99% reduction in the computational effort required for Nodeinfo and an F-Measure score of up to 100% in the identification of regions of the event log which contain alerts. Our work demonstrates a practical application of employing MTI on a real world data set using an alert detection framework that is currently in production use in a major government run national laboratory.